Researching Complex Interventions in Health: The State of the Art

S Open Access Researching Complex Interventions in Health: The State of the Art Exeter, UK. 14-15 October 2015 Published: 4 April 2016 KEYNOTE PRESENTATIONS K1 Researching complex interventions: the need for robust approaches Peter Craig ([email protected]) MRC/CSO Social and Public Health Sciences Unit (SPHSU), University of Glasgow, 200 Renfield Street, Glasgow G2 3QB, UK BMC Health Services Research 2016, 16(Suppl 1):K1 Since the revised MRC guidance on the development and evaluation of complex interventions was published in 2008, much progress has been made in developing and defining good practice, and there is an increasingly detailed and comprehensive set of guidance available to help researchers and funders make the right choices. But progress is patchy, and evidence continues to accumulate of waste in research. Robust approaches are needed, that combine good practice across all stages of the evaluation process, from the initial choice and framing of research questions, through to implementation and translation of evidence. The elements of a robust approach are mostly well-known, but much more rarely are they combined into a coherent package. The talk will consider why this is so, and what can be done to improve matters. K2 Complex intervention studies: an important step in developing knowledge for practice Ingalill Rahm-Hallberg ([email protected]) Lund University, PO Box 117, 221 00 Lund, Sweden BMC Health Services Research 2016, 16(Suppl 1):K2 This presentation will provide the history of the “Complex Interventions in Health” book. The development of the book took off in the European Academy of Nursing Science (EANS) summer school. For more than 12 years the EANS has gathered doctoral students from all over Europe in a three year programme. The content developed over time, inspired by the critique stating that nursing research was heavily dominated by descriptive, crosssectional or qualitative research not really informing practice. This lead to building a curriculum based on the MRC guidance on complex interventions and the teaching programme became truly successful among students. The road to developing the content for a book was short. Professor Richards and I played around on the black board and after that it has been a success-story. So many authors delivering on time! However, rightfully we have been told that research in health is more than researching complex interventions. I cannot agree more, it is only one step, but a very important one. Health research, as any research with aspirations for informing practice, needs to be carried out systematically and programmatically, using a variety of designs and methods. It is helpful to think of knowledge development as being a stepwise process starting off with discovery and once possible going into the phase © 2016 The Author(s). Open Access This artic International License (http://creativecommons reproduction in any medium, provided you g the Creative Commons license, and indicate if (http://creativecommons.org/publicdomain/ze of evaluation and once solid knowledge is obtained it is about implementation in practice. K3 Public and patient involvement in research: what, why and how? Nicky Britten ([email protected]) University of Exeter Medical School, South Cloisters, St Luke’s Campus, Exeter EX1 2LU, UK BMC Health Services Research 2016, 16(Suppl 1):K3 In this presentation I will explain what PPI is, what it isn’t, and how it differs from engagement, as there is some misunderstanding. There is a range of both ethical and pragmatic reasons for involving members of the public and patients in research, depending on the different values and perspectives of those involved. I will provide practical examples of PPI at different stages of the research cycle: identifying and prioritising research questions; developing research proposals; doing research; analysing research data; disseminating and implementing findings; as well as PPI in systematic reviewing and operational research. Drawing on my own research, I will use the particular example of the Diabetes Intervention for Agenda Trial (DIAT) to illustrate PPI in a trial from start to finish. I will present a theoretical framework for characterising PPI which paradoxically is of practical value in reflecting on PPI practice. Finally I will present the Public Involvement Impact Assessment Framework (the PiiAF) for helping to evaluate the various intended and unintended impacts of PPI. K4 Mixed methods in health service research – where do we go from here? Gunilla Borglin ([email protected]) Department of Caring Science, Malmö University, SE-205 06 Malmö, Sweden BMC Health Services Research 2016, 16(Suppl 1):K4 In the 21st century, it is more obvious than ever before that health services research and its practitioners exist in an extremely complex contextual environment. Consequently, at the heart of understanding how to deliver an evidence base for safe and effective care in a setting characterised by multifaceted health care demands, is the realisation that no one research method in isolation will suffice. This realisation, together with the REWARD group’s contention that 85 % of health service research conducted – and most importantly mainly funded by public money – is ‘waste’, is one of the reasons why frameworks such as the MRC guidance on complex interventions, and mixed methods designs are subject to intensified attention. This plenary address aims to discuss the contribution of mixed methods to researching complex interventions within the MRC framework and to consider its place in contemporary health service research. The MRC framework for researching complex interventions has highlighted the le is distributed under the terms of the Creative Commons Attribution 4.0 .org/licenses/by/4.0/), which permits unrestricted use, distribution, and ive appropriate credit to the original author(s) and the source, provide a link to changes were made. The Creative Commons Public Domain Dedication waiver ro/1.0/) applies to the data made available in this article, unless otherwise stated. BMC Health Services Research 2016, Volume 16 Suppl 1 Page 2 of 5 importance of research designs including both qualitative and quantitative approaches. However when reviewing the literature prior to writing my chapter in our recent textbook, it was clear that designing and conducting truly mixed methods research presents health service researchers with a number of challenges. Predominant amongst these are the selection of designs from suggested typologies, methodological reporting, critical evaluation and most importantly ensuring that the core feature in true mixed methods (as opposed to multi-methods) designs – analytical and methodological integration – is implemented. To be able to conduct mixed methods approaches that are rigorously designed, logically executed, and transparently reported, we need to move to a position where funders, researchers, journal editors and research consumers demand methodological integration of methods and data from study outset, rather than as a mere afterthought in the discussion section of research reports. SPEAKER PRESENTATIONS S1 Exploring complexity in systematic reviews of complex interventions Sascha Köpke, Jane Noyes, Jackie Chandler University of Lübeck, Germany; Bangor University, UK; Cochrane Collaboration, Oxford, UK Correspondence: Gabriele Meyer ([email protected] ) – Martin Luther University Halle-Wittenberg, Magdeburger Str. 8, D-06112 Halle (Saale), Germany BMC Health Services Research 2016, 16(Suppl 1):S1 There is a lot of discussion about the most adequate approaches to perform systematic reviews and meta-analyses of complex interventions. Reviewers need to broaden their view and to have a detailed look at all available evidence for the whole process of developing and evaluating complex interventions. Decisions about approaches should acknowledge intervention aims. Decisions about the scope and methods of the systematic review depend on the available information, which should be assessed in view of clinicians’ and patients’ needs. Important decisions include the type of evidence to be searched and the choice of methods to describe or synthesise it. This will usually require a combination of different study types including quantitative and qualitative data. Transparent reporting is a key issue to allow readers to apply the findings to specific contexts. S2 Can complex health interventions be optimised before moving to a definitive RCT? Strategies and methods currently in use Sara Levati ([email protected]) NMAHP Research Unit, Glasgow Caledonian University, UK BMC Health Services Research 2016, 16(Suppl 1):S2 RCTs are recognised as the ‘gold standard’ methodology in the evaluation of complex interventions, however, they often fail to detect whether the lack of intervention effect is due to sub optimal design, implementation failure or genuine ineffectiveness. Given the current financial constraints upon health services research, it is increasingly important to define pre-trial optimisation methods that can give indications on how the intervention works and help maximise chances for the intervention to be effective. This scoping review provides a map of the health literature on pre-trial strategies aimed at optimising complex interventions before a RCT and a snapshot of the methods currently used. Scholarly literature was searched using MEDLINE, CINAHL, AMED, PsycINFO and ProQuest Nursing & Allied Health Source for papers published between January 2000 and March 2015 available in English language. The literature search identified 3940 unique references, 27 of which met the inclusion criteria. Optimisation strategies explored the feasibility and acceptability of the intervention to patients and healthcare professionals, estimated the effectiveness and cost-effectiveness of different combinations of components and identified potential barriers to implementation. Large variations in the methods adopted were noted across studies, including interviews and focus groups with interventions’ providers and receivers, experts consultations, economic modelling, small uncontrolled pilot studies and evaluation questionnaires. Overall, there is the potential for optimisation strategies to detect, in a costeffective way, those interventions and components that are likely to fail or show little effect if implemented in a full-scale RCT. S3 A systematic approach to develop theory based implementation interventions Anne Sales ([email protected]) School of Nursing, University of Michigan, USA and Center for Clinical Management Research, VA Ann Arbor Healthcare System, USA BMC Health Services Research 2016, 16(Suppl 1):S3 In this presentation, I will briefly review the large landscape of frameworks and theories in implementation research, and suggest an approach to designing implementation interventions that builds on current widely cited frameworks to develop theory-based implementation interventions. S4 Pilot studies and feasibility studies for complex interventions: an introduction Lehana Thabane, Lora Giangregorio McMaster University, Canada; University of Waterloo, Canada Correspondence: Lehana Thabane ([email protected]) – McMaster University, Canada BMC Health Services Research 2016, 16(Suppl 1):S4 Pilot studies for phase III trials—which are comparative randomized trials designed to provide preliminary evidence on the clinical efficacy of an intervention—are routinely performed in many clinical areas. Also commonly known as “feasibility”, “dress rehearsal” or “vanguard” studies, they are designed to assess recruitment potential; to assess the feasibility of international collaboration or coordination for multicentre trials; to increase clinical experience with the study intervention for the phase III trials. They are the best way to assess feasibility of a large, expensive full-scale study, and in fact are an almost essential pre-requisite (Thabane et al (BMC Medical Research Methodology 2010, 10:1). Conducting a pilot prior to the main study can enhance the likelihood of success of the main study and potentially help to avoid doomed main studies. The presentation will cover some key aspects of pilot studies for phase III trials of complex interventions including: 1) reasons for conducting a pilot study; 2) misconceptions about pilot studies; 3) criteria for evaluating the success of a pilot study; 4) differences and similarities between pilot and feasibility studies; and 5) a brief update on the development of CONSORT extension for pilot trials. S5 What can be done to pilot complex interventions? Nancy Feeley, Sylvie Cossette Ingram School of Nursing, McGill University, and Centre for Nursing Research and Lady Davis Institute, Jewish General Hospital, Montreal, Québec, Canada; Faculty of Nursing, University of Montreal, and Montreal Heart Institute Research Center, Montreal, Québec, Canada Correspondence: Nancy Feeley ([email protected]) – Ingram School of Nursing, McGill University, and Centre for Nursing Research and Lady Davis Institute, Jewish General Hospital, Montreal, Québec, Canada BMC Health Services Research 2016, 16(Suppl 1):S5 In a pilot or feasibility study, the feasibility and acceptability of many of the key features of a complex intervention can be examined. The main research questions are: Can the intervention be provided as planned? Is the intervention acceptable to participants? Certain features (e.g., content, sequence, dose, timing, mode of delivery) will be more important to scrutinize than others, depending on the particular BMC Health Services Research 2016, Volume 16 Suppl 1 Page 3 of 5 challenges anticipated and the questions that remain at the end of intervention development. This presentation will examine which intervention features might be examined in a pilot study, and describe examples of studies that have done so. In addition, special issues such as contamination and co-intervention will be discussed. S6 Using feasibility and pilot trials to test alternative methodologies and methodological procedures prior to full scale trials Rod Taylor ([email protected]) University of Exeter Medical School, South Cloisters, St Luke’s Campus, UK BMC Health Services Research 2016, 16(Suppl 1):S6 Feasibility and pilot studies play a key role in health research, in providing information for the planning of full scale randomised controlled trials. Medical Research Council guidance for development and evaluation of complex interventions states that pilot and feasibility studies are essential in the development and testing of an intervention prior to a large-scale evaluation. Before committing investment in costly and time-consuming full scale trials, funding bodies increasingly demand that investigators provide evidence from pilot/feasibility studies addressing the question: “Can this full scale trial be done?” This presentation will: (i) identify the common methodological uncertainties in the design and execution of full scale trials; (ii) review approaches to testing these methodological uncertainties in feasibility/pilot studies; and (iii) discuss the design and selection of feasibility and pilot studies to best address these uncertainties. These concepts will be illustrated by reference to recent feasibility/pilot studies. S7 A mixed methods feasibility study in practice Jacqueline Hill, David A Richards, Willem Kuyken Mood Disorders Centre, School of Psychology, University of Exeter, UK; University of Exeter Medical School, UK; Department of Psychiatry, University of Oxford, UK Correspondence: Jacqueline Hill ([email protected]) – Mood Disorders Centre, School of Psychology, University of Exeter, UK BMC Health Services Research 2016, 16(Suppl 1):S7 Depression is set to become the third biggest cause of the global burden of disease by 2030 yet access to treatment is poor. To maximise access, worldwide psychological therapies for depression are delivered using a system called stepped care. However, we do not know if this system achieves similar patient benefit for less cost compared with alternatives. A fully-powered clinical trial of stepped care is required but currently prevented by a number of uncertainties. This presentation will describe how we chose to answer these in a single feasibility study encompassing a pilot randomised controlled trial and embedded qualitative interviews. In particular, the presentation will focus on the innovative use of integrated mixed methods analysis to explore how patients’ views of stepped care and the number of treatment sessions they attend relate. In this way, we will illustrate how it is possible to interweave quantitative and qualitative data in original ways as part of a feasibility study to address multiple clinical, procedural and methodological uncertainties. S8 Non-standard experimental designs and preference designs Louise von Essen ([email protected]) Department of Public Health and Caring Sciences, Uppsala University, Sweden BMC Health Services Research 2016, 16(Suppl 1):S8 Randomized controlled trials (RCT) have been accepted as the ‘gold standard’ design for evaluating the effectiveness of a single intervention such as a drug. However, recruitment to clinical trials may be affected by the choice of intervention that participants might make, if they were allowed to choose, and by whether they actually receive their preferred intervention. These effects cannot be estimated in the RCT design. Additionally the RCT runs counter to the current emphasis on patient choice which is the cornerstone in many governments’ current health strategies and where preferences or ethical objections to an RCT exist, alternatives that replicate more closely the behaviors in real world health care rather than conform to standard trial design should be considered. Citizen participation in research is advocated by governments, research councils, and other funding bodies and as the public, patients, and significant others become more involved in research activities they may be unwilling to be passive participants of a research randomization process. Non-standard and preference designs go some way towards addressing the problems of low recruitment and retention rates and non-implementation of results of clinical research. Advantages and challenges e.g. with regard to internal and external validity with non-standard and preference designs such as the comprehensive cohort design, pre-randomized design, cohort multiple randomized controlled trial, and two-stage randomized design will be presented at the conference. S9 Evaluation gone wild: using natural experimental approaches to evaluate complex interventions Andrew Williams ([email protected] or [email protected]) Farr Institute, Scotland, and the Scottish Collaboration for Public Health Research and Policy, 20 West Richmond Street (East Wing), Edinburgh EH8 9DX, UK BMC Health Services Research 2016, 16(Suppl 1):S9 Any outcome evaluation of an intervention needs to address the following questions: 1. Did change occur? 2. Was the change due to the intervention? 3. Was the change significant/meaningful? (Sanson-Fisher et al, 2014) [1] With the ability to calculate the unbiased attributable effects on an intervention the randomised controlled trial (RCT) is seen as the ‘gold standard’ method for answering all three questions. However, as complex interventions are recognised as events within systems, they are rarely tame enough for RCTs to be feasible (Hawe et al., 2009) [2]. The natural experimental approaches (NEAs) permit quasiexperimental evaluation which can answer the three questions when the intervention is wilder, particularly when routinely collected outcome data are available (Craig et al., 2011) [3]. Robust NEAs rely on quasi-randomised identification (instrumental variable, regression discontinuity or propensity score matching) of a control (counterfactual) population to compare with the intervention population. Subsequently, taking a difference in differences or controlled before-after approach permits the calculation of an effect attributable to the intervention addressing the three questions. Quasirandomised scenarios often occur by chance (naturally, not designed as part of an evaluation) within complex interventions allowing valuable prospective or retrospective evaluation. For example, criterionbased intervention eligibility permitting a regression discontinuity design. Although not faultless, NEAs offer an often missed opportunity to evaluate complex interventions.